SConstruct : Add Windows libraries for Cycles GPU rendering #5761
Conversation
|
I added a commit to update to the |
|
Thanks Eric! Unfortunately I've not had much success with this Windows CI build finding the CUDA or Optix devices on my workstation, only the CPU is reported in To rule out any driver related mischief I checked both Blender 4.0.2 and Alex's earlier Windows GPU Gaffer builds and they both find the GPU devices, so maybe it's related to the CI build environment if local builds are working on your machine? |
|
I was a bit concerned at the time when removing the devices method, as this function isn't re-entrant in the past. It might be the case where it is re-entrant in Linux but not in Windows. Worth a try to revert this change and see if devices appear. |
|
I cleared out all my caches and tried again with the artifacts from the build last night. Cycles picked up CUDA but not Optix. I think I sorted out Optix, no idea how it was getting found earlier, but I fixed an error in my dependencies build and confirmed it's working for me with caches cleared again. So I'm up to getting Optix and CUDA again on my workstation. I wonder if it's a versions issue? I have CUDA 11.8 and Optix 7.7.0 installed on my system. |
|
I kicked off a new build of CI with my dependencies package fix. I'll check that again once it's done. |
|
The latest CI build (dated March 3 instead of March 2 in the archive filename) gives me both Optix and CUDA now. So partial success! I'll give it a try on my laptop as well, which hasn't seen any development work in this area and can be considered fresh for this purpose. |
|
My laptop won't work unfortunately because it doesn't have an NVIDIA card. Might need to rely on @murraystevenson as to test again with the new CI build from today. |
Ah, in my case I didn't have the CUDA toolkit installed. Installing CUDA 11.8 then makes the CUDA and Optix devices available on my workstation with the latest CI build, but as mentioned previously this isn't required for Blender or Alex's builds... |
|
I didn't realize it's possible to run the GPU devices without the toolkit installed. I'll look into what I'm missing. |
|
Yep the toolkit isn't needed if a few conditions are met. The |
ericmehl
force-pushed
the
WindowsCyclesCudaOptix
branch
from
cf39e3f
to
6381192
Compare
|
Ok, I have a pretty good handle on what's been tripping us up getting the GPU devices on Windows. The problem at the center is we weren't getting the right path set in Cycles' The patch that Alex put into Cycles itself was getting the executable path ( But investigating that showed that the Cycles
In my latest commit, I register the I also removed the patch from GafferDependencies Cycles since it should not find an empty Hopefully when the CI build goes through, we can test again and find all the GPU devices... |
|
That explanation seems completely plausible, and it's great that we can remove the patch from GafferHQ/dependencies. The only problem is that my NVIDIA device still isn't being found with the latest build artifacts from the PR. I've confirmed that it is found by Blender (1.0.2 and 4.1). And I've also switched I'm using NVIDIA driver 536.67, with CUDA 12.2.128 driver. |
|
The new build artifacts after the latest push are working for me now. 🎉 |
|
The latest push is also working for me on Windows. 🥳 |
There was a problem hiding this comment.
Good job hunting this down @ericmehl. I'm definitely happy to go with this workaround for now, but I mentioned inline one alternative approach that might be better, assuming it works on Windows...
| // Must be initialized to ensure the statically linked Cycles module | ||
| // we are linked to can find the binaries for GPU rendering. | ||
| ccl::path_init( IECoreCycles::cyclesRoot() ); |
There was a problem hiding this comment.
In theory, anything in IECoreCycles::init() could need this special duplicate treatment, right? And we're just lucky that at the moment there's just some additional logging bits and bobs in there?
This got me wondering - do we even need to link the GafferCycles module to Cycles at all? It already links to libGafferCycles, so can't it get the Cycles symbols from there? I gave that a go here and it seemed to compile and run on Linux at least. Might be worth giving that a shot on Windows too? It'd certainly be less fragile if it works....
There was a problem hiding this comment.
We talked about this offline and tried a few things to remove the need for linking the library in both modules, but unfortunately none worked. Without the linking the Python module to the various Cycles libs, there are 11 unresolved symbols. We also tried linking GafferCycles with /WHOLEARCHIVE and variants on /WHOLEARCHIVE:cycles_util but none worked, instead generating different unresolved symbols within GafferCycles.
Leaving as-is for now, with the hope that some day there will be a dynamic Cycles library we can link to.
| @@ -47,6 +47,10 @@ IECORE_POP_DEFAULT_VISIBILITY | |||
| namespace IECoreCycles | |||
| { | |||
|
|
|||
| /// Returns the value of the `CYCLES_ROOT` environment variable. | |||
| /// Throws if the variable is not set. | |||
There was a problem hiding this comment.
The implementation is actually returning "" rather than throwing.
There was a problem hiding this comment.
Right! I had debated which to do and that comment got stale. Fixed in b5834f6
Because we're linking to static Cycles libraries, any static variables will be local to that instance of the linked library. By calling `IECoreCycles::init()`, we were registering the Cycles root directory with the library linked to `IECoreCycles` / `GafferCycles`. That meant `GafferCycles` _module_ was not registering the Cycles root, and GPU devices would fail to initialize because it could not find their binary compilers. Registering the Cycles root in both modules allows all Cycles library instances to find the files it needs.
ericmehl
force-pushed
the
WindowsCyclesCudaOptix
branch
from
af1de8d
to
13db05a
Compare
This adds support for rendering with CUDA and Optix in Cycles on Windows. I'm not sure it will build successfully until the dependencies have been updated to
8.0.0a10. But when that's ready, this can be converted out of draft status.Checklist